Skip to content

Conversation

marbemac
Copy link

@marbemac marbemac commented Oct 5, 2025

Fix optimistic inserts not showing in live queries before sync completes.

Related Discord discussion: https://discord.com/channels/933657521581858818/1374668288549847071/1423800713221505145

The Problem

We ran into an issue where if you:

  1. Create an Electric collection (or any collection that doesn't sync immediately)
  2. Insert data optimistically
  3. Create a live query from that collection

The live query would return empty results until Electric finished syncing, even though the optimistic data was sitting right there in the collection.

This was breaking our use case where we create message collections on-demand (per-thread partitioning) and insert the first message before Electric has synced anything.

The Fix

The issue was in allCollectionsReadyOrInitialCommit() - it was refusing to run the graph on collections that were still idle or loading, even if they had data.

Added a simple check: if collection.size > 0, let the graph run. This allows processing optimistic data while waiting for sync to complete.

Tested in our app and confirmed the messages now show up instantly instead of waiting for Electric to sync.

Disclaimer: I am NOT an expert on this codebase.. unsure if there are downstream ramifications to this tweak, but all of the tests pass...


Side note: the PR template tells the user to verify they have followed the contributing guide @ https://github.com/TanStack/db/blob/main/CONTRIBUTING.md - which no longer exists.

inserts  before sync completes

Signed-off-by: Marc MacLeod <[email protected]>
Copy link

changeset-bot bot commented Oct 5, 2025

🦋 Changeset detected

Latest commit: 294de56

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 12 packages
Name Type
@tanstack/db Patch
@tanstack/angular-db Patch
@tanstack/electric-db-collection Patch
@tanstack/query-db-collection Patch
@tanstack/react-db Patch
@tanstack/rxdb-db-collection Patch
@tanstack/solid-db Patch
@tanstack/svelte-db Patch
@tanstack/trailbase-db-collection Patch
@tanstack/vue-db Patch
todos Patch
@tanstack/db-example-react-todo Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Signed-off-by: Marc MacLeod <[email protected]>
@KyleAMathews
Copy link
Collaborator

This could work — I think it'd be unexpected to many people as you'd show the optimistic mutation right away and then the synced data would pop in right after — typically I think people would preload the collection before going to the route & allowing optimistic mutations.

I think you're use case is you only create the collection when someone does a mutation and you know in this case it's empty? Could you just preload that empty creation when you enter the route?

@marbemac
Copy link
Author

marbemac commented Oct 6, 2025

I think it'd be unexpected to many people as you'd show the optimistic mutation right away and then the synced data would pop in right after — typically I think people would preload the collection before going to the route & allowing optimistic mutations

Yeah I think having collection loaded on screen before rendering ui that does optimistic inserts etc is def the typical use case. But the change here shouldn't really affect that case right? As far as I can tell, this change omostlynly affects folks that are optimistically inserting before sync is started - and in this case the expectation (at least my expectation hah) is that the optimistically inserted data is immediately available.

I think you're use case is you only create the collection when someone does a mutation and you know in this case it's empty? Could you just preload that empty creation when you enter the route?

Our use case is that we create a new shape (collection) per chat thread. So the user is on the home screen:

  1. types something into the chat box & presses enter. this runs a tanstack db transaction that:
    a. optimistically inserts a thread record with id thread_xyz into the threads collection (there is only one of this collection - it's global)
    b. creates a new messages collection for thread thread_xyz, and optimistically inserts the user's message into it
  2. immediately navigates to the route for the thread, e.g. /chat/thread_xyz
  3. the thread screen renders a messages react component that creates a live query on that newly created messages collection. this live query kicks off the syncing
const messagesCollection = messageCollectionForThread('thread_xyz');

const { data, isReady } = useLiveQuery(
  q => q.from({ m: messagesCollection }).orderBy(({ m }) => m.createdAt, 'desc')
);

We're doing all of this in order to partition the messages queries, rather than always fetching lifetime history of all messages across all threads.

I thought that this was the recommended way to partition data atm, but I'm totally open to alternative approaches.

@KyleAMathews
Copy link
Collaborator

There was another use case that came up recently where you wouldn't want to wait for sync. When you have a left join and you don't want to block the initial render on the second collection loading.

Both and that and what you're doing feel like an opt in thing though as it's a bit surprising and perhaps unwanted otherwise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants